Skip to content

perf(producer): stage frames via a junction before copying on Windows#2314

Open
sidorovanthon wants to merge 2 commits into
heygen-com:mainfrom
sidorovanthon:fix/producer-symlink-copy-fallback
Open

perf(producer): stage frames via a junction before copying on Windows#2314
sidorovanthon wants to merge 2 commits into
heygen-com:mainfrom
sidorovanthon:fix/producer-symlink-copy-fallback

Conversation

@sidorovanthon

@sidorovanthon sidorovanthon commented Jul 13, 2026

Copy link
Copy Markdown

What

Rebased onto current main and re-scoped. When linkOrCopyFrameDir can't create a plain symlink for an extracted-frames dir, it now tries a junction before falling back to the full recursive copy.

Why

main already degrades a rejected symlink (EPERM/EACCES/UNKNOWN — Windows without Developer Mode/Administrator) to cpSync(recursive). That copy is correct but, as the code comment itself notes, measurably slower — every frame of every video is physically copied into the compiled dir.

A Windows junction is a directory link that needs neither Developer Mode nor Administrator. So the common no-privilege case can stage the frame dir at link speed instead of copying it. The copy stays exactly where it was, but now only as the last resort for mounts that support no links at all (SMB/NFS/exFAT).

How

linkOrCopyFrameDir becomes three tiers:

  1. Plain symlink — unchanged cheap default (POSIX, Windows with Developer Mode).
  2. Junction — attempted only after a plain symlink is rejected with EPERM/EACCES/UNKNOWN. On POSIX Node ignores the "junction" type, and we only reach this tier on a privilege error POSIX never raises, so POSIX behavior is unchanged. A junction needs a local absolute target and doesn't exist on SMB/NFS/exFAT, so a capability error here (EPERM/EACCES/UNKNOWN/EINVAL/ENOSYS/EOPNOTSUPP/ENOTSUP, see isSymlinkCapabilityError) drops to tier 3. A non-capability junction error (e.g. ENOSPC) still propagates.
  3. Recursive copy — the existing last resort, with the one-time degrade notice.

MaterializeFileSystem.symlinkSync gains an optional type?: "junction" parameter; the default fs-backed implementation already forwards it, and the only other staging path (materializeSymlinks, distributed plan()) is untouched.

Test plan

renderOrchestrator.test.tsmaterializeExtractedFramesForCompiledDir suite:

  • junction fast-path: plain symlink throws EPERM, junction succeeds → cpSync is not called, frames remapped exactly as in the symlink happy path.

  • copy only when both fail: plain symlink and junction both throw EPERM → plain then junction attempted (asserted via the recorded type argument), then cpSync(recursive).

  • UNKNOWN variant unchanged (both link attempts fail → copy).

  • non-capability junction error (ENOSPC from the junction attempt) → rethrown, no copy.

  • Unit tests added/updated (renderOrchestrator.test.ts: 145 passed / 0 failed via vitest)

  • bunx oxlint / bunx oxfmt clean; tsc --noEmit clean; lefthook pre-commit (lint/format/fallow/typecheck) green

  • Documentation updated (if applicable) — behavior/perf-only change, no docs affected

🤖 Generated with Claude Code

@miguel-heygen

Copy link
Copy Markdown
Collaborator

@sidorovanthon rebase main pls, after that I'll review again

@sidorovanthon sidorovanthon force-pushed the fix/producer-symlink-copy-fallback branch from 9d5eb34 to abf3c6e Compare July 13, 2026 09:46
@sidorovanthon sidorovanthon changed the title fix(producer): fall back to copying frames when extract-cache symlink fails perf(producer): stage frames via a junction before copying on Windows Jul 13, 2026
@sidorovanthon

Copy link
Copy Markdown
Author

@miguel-heygen rebased onto current main and re-scoped the PR.

The original diff predated your linkOrCopyFrameDir symlink→copy fallback and largely duplicated it. On a fresh main that fallback is already there, so I dropped the duplication and kept only the part main doesn't cover: a junction attempt between the symlink and the copy.

A Windows junction needs neither Developer Mode nor Administrator, so the common no-privilege case now stages the frame dir at link speed instead of copying every frame (your own comment flags the copy as "measurably slower"). The copy stays as the last resort for mounts with no link support at all (SMB/NFS/exFAT), where even a junction fails with a capability error; a non-capability junction error (e.g. ENOSPC) still propagates. POSIX behavior is unchanged — that tier is only reached on a privilege error POSIX never raises.

Title/description updated to match. Ready for another look.

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The junction implementation is currently bypassed by the exact local Windows path this PR is intended to optimize. renderOrchestrator.ts passes materializeSymlinks: shouldCopyExtractedFrames(process.platform), and shouldCopyExtractedFrames("win32") is true; stageExtractedFrameDirOnce(..., true) calls cpSync(recursive) directly and never reaches linkOrCopyFrameDir, so neither the plain-symlink nor junction tier runs. A fresh 0.7.58 Windows field report reproduced the consequence with one 305.453s video: all 9,164 JPEGs were extracted, then the process exited while materializing the cache into compiledDir. Please close the class in this PR by routing local Windows staging through the link/junction/copy ladder (while keeping distributed-plan materialization as a real copy), and add an orchestrator-level regression proving the actual win32 local-render call does not request eager copy. The helper-only tests currently pass while the production caller still selects the old path.

Anton Sidorov aka anticodeguy and others added 2 commits July 15, 2026 08:48
`linkOrCopyFrameDir` already degrades a rejected symlink to a full
recursive copy of every extracted frame (EPERM/EACCES/UNKNOWN — Windows
without Developer Mode). Copying is measurably slower than linking.

Insert a junction attempt between the symlink and the copy: a Windows
junction needs neither Developer Mode nor Administrator, so the common
no-privilege case now stages the frame dir at link speed instead of
copying. On POSIX the "junction" type is ignored by Node, and that tier
is only reached on a privilege error POSIX never raises, so behavior
there is unchanged. The copy fallback stays as the last resort for
mounts with no link support at all (SMB/NFS/exFAT), where even a
junction fails with a capability error; a non-capability junction error
(e.g. ENOSPC) still propagates.

`MaterializeFileSystem.symlinkSync` gains an optional `type?: "junction"`
param; the default fs-backed impl already forwards it.

Tests: junction fast-path (symlink EPERM → junction succeeds → no copy);
copy only when both symlink and junction fail; non-capability junction
error propagates. 145 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WiwMWMe25vjrDaiEsVFwQe
…ction/copy ladder

The junction fast-path was dead code on the exact platform it targeted.
The local render invoked the extract stage with
`materializeSymlinks: shouldCopyExtractedFrames(process.platform)`, and
`shouldCopyExtractedFrames("win32")` is `true`, so `stageExtractedFrameDirOnce`
called `cpSync(recursive)` directly and never reached `linkOrCopyFrameDir` —
neither the plain-symlink nor the junction tier ran. A 0.7.58 Windows field
report reproduced the consequence: 9,164 JPEGs extracted, then the process
exited while materializing the cache into `compiledDir`.

Fix the class, not the symptom:

- `renderOrchestrator.ts`: the local in-process render passes
  `materializeSymlinks: false`, routing win32 through the
  symlink -> junction -> copy ladder. A junction needs neither Developer Mode
  nor admin, so the common no-privilege case stages at link speed; the
  recursive copy stays as the ladder's last resort (SMB/NFS/exFAT).
- Delete `shouldCopyExtractedFrames`: the platform->copy coupling is exactly
  what bypassed the ladder, and the junction tier now owns the win32 privilege
  case. Removing it makes the bypass structurally impossible to reintroduce.
- Tighten the `materializeSymlinks` JSDoc: eager copy is reserved for
  distributed `plan()` (self-contained planDir); do NOT recouple it to
  `process.platform`. `plan.ts` is unchanged.

Regression proven at the stage the orchestrator actually calls
(`runExtractVideosStage`), not a helper: the local path does not request an
eager copy (flag `false` or omitted), and only the distributed path does.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LNERhhFUhVkVBn3SksgeZq
@sidorovanthon sidorovanthon force-pushed the fix/producer-symlink-copy-fallback branch from abf3c6e to 33e0789 Compare July 15, 2026 03:05
@sidorovanthon

sidorovanthon commented Jul 15, 2026

Copy link
Copy Markdown
Author

@miguel-heygen thanks - you were exactly right, the junction ladder was dead code on the very platform it targeted. Rebased onto current main and closed the class in this push (33e0789b0).

Root cause. The local render invoked the extract stage with materializeSymlinks: shouldCopyExtractedFrames(process.platform), and shouldCopyExtractedFrames("win32") is true. stageExtractedFrameDirOnce(..., true) calls cpSync(recursive) directly, so neither the plain-symlink nor the junction tier in linkOrCopyFrameDir ever ran. The helper tests passed while the production caller still selected the old path - precisely as you noted.

Fix:

  1. renderOrchestrator.ts - the local in-process render now passes materializeSymlinks: false, routing win32 through the symlink → junction → copy ladder. A junction needs neither Developer Mode nor admin, so the common no-privilege case stages at link speed; the recursive copy stays as the ladder's last resort (SMB/NFS/exFAT).
  2. Deleted shouldCopyExtractedFrames - the platform → copy coupling is exactly what bypassed the ladder, and the junction tier now owns the win32 privilege case. Removing it makes the bypass structurally impossible to reintroduce; its JSDoc on materializeSymlinks now spells out "distributed plan() only — do not recouple to process.platform."
  3. plan.ts unchanged - distributed staging keeps materializeSymlinks: true (a planDir must be self-contained; symlinks/junctions don't survive S3/GCS).
  4. Orchestrator-level regression - added at runExtractVideosStage (the stage the orchestrator actually calls, not a helper): the local path does not request an eager copy (flag false or omitted → materializeExtractedFramesForCompiledDir reaches the ladder), and only the distributed path does. Existing junction fast-path helper test already proves false + EPERM → junction, no copy - so the full chain is covered.

Behaviorally a strict improvement: the ladder's tier 3 is the same cpSync(recursive), so the win32 worst case is identical to today, best case is a privilege-free junction.

Verification (Windows): producer unit lane 306 passed; renderOrchestrator.test.ts 143/143 (junction ladder intact); extractVideosStage.test.ts 7/7 incl. 3 new; tsc --noEmit, oxlint, oxfmt, test-classification all clean. (One unrelated pre-existing failure in hyperframeRuntimeLoader.test.ts - a Windows path-separator assertion - reproduces without this change.)

Ready for another look.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants